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Homeomorphisms which are Dehn twists on the boundary 

Darryl McCullough 



A homeomorphism of a 3 -manifold M is said to be Dehn twists on the boundary 
when its restriction to dM is isotopic to the identity on the complement of a 
collection of disjoint simple closed curves in dM . In this paper, we give various 
results about such collections of curves and the associated homeomorphisms. In 
particular, if M is compact, orientable, irreducible and dM is a single torus, and M 
admits a homeomorphism which is a nontrivial Dehn twist on dM, then M must 
be a solid torus. 

57M99; 57R50 



Introduction 

A homeomorphism h of a compact 3 -manifold M is said to be Dehn twists on the 
boundary when its restriction to dM is isotopic to the identity on the complement of a 
collection of disjoint simple closed curves in dM. If this collection is nonempty, and 
the restricted homeomorphism is not isotopic to the identity on the complement of any 
proper subset of the collection, then we say that h is Dehn twists about the collection. 
The restriction of h to dM is then isotopic to a composition of nontrivial Dehn twists 
about the curves, where for us a Dehn twist may be a power of a "single" Dehn twist. 
Note that the minimality condition implies that each curve in the collection is essential 
in dM, and no two of them are isotopic in dM. Our first main result gives strong 
restrictions on the collection of curves. 

Theorem 1 Let M he a compact orientable 3 -manifold which admits a homeomor- 
phism which is Dehn twists on the boundary about the collection Ci, . . . , C„ of simple 
closed curves in dM. Then for each i, either Ci bounds a disk in M, or for some j ^ i, 
Ci and Cj cobound an incompressible annulus in M. 

Our second main result gives structural information about such homeomorphisms. It 
refers to Dehn twists about disks and annuli in M, whose definition is recalled in 
Section 1. 
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Theorem 2 Let M he a compact orientable 3 -manifold which admits a homeomor- 
phism which is Dehn twists on the boundary about the collection Ci , . . . , C„ of simple 
closed curves in dM . Then there exists a collection of disjoint imbedded disks and 
annuli in M, each of whose boundary circles is isotopic to one of the C,, for which 
some composition of Dehn twists about these disks and annuli is isotopic to h on dM. 

That is, h must arise in the most obvious way, by composition of Dehn twists about a 
collection of disjoint annuli and disks with a homeomorphism that is the identity on the 
boundary. 

Theorem 1 and Theorem 2 yield strong statements for specific classes of manifolds. For 
the case when M is a compression body, examined in Section 3, a homeomorphism 
which is Dehn twists on the boundary is actually isotopic to a product of Dehn twists 
about disjoint annuli and disks. This appears in Oertel [10] for the case when M is a 
handlebody. Another application is the following: 

Corollary 0.3 Let M be a compact orientable irreducible 3 -manifold with boundary 
a torus. If M admits a homeomorphism which is a Dehn twist on dM, then M is a solid 
torus and the homeomorphism is isotopic to a Dehn twist about a meridian disk. 

In particular, the only knot complement in S^ (indeed, the only irreducible complement 
of a knot in any closed orientable 3 -manifold) admitting a homeomorphism which is a 
nontrivial Dehn twist on the boundary is the trivial knot. 

Proof of Corollary 0.3 By Theorem 1, C bounds a disk in M. Since M is irreducible, 
this implies that M is a solid torus. By Theorem 2, there is a Dehn twist about the 
meridian disk which is isotopic on dM to the original homeomorphism, and since any 
homeomorphism which is the identity on the boundary of a solid torus is isotopic to the 
identity, this Dehn twist and the original homeomorphism must be isotopic. n 

It appears that most of our results could be extended to the nonorientable case, adding 
the possibility of Dehn twists about Mobius bands in Theorem 1 , Theorem 2 and 
Corollary 3.1, but the proof of Lemma 1 .2 would require the more elaborate machinery 
of uniform homeomorphisms, found in McCullough [7] or Chapter 12 of Canary and 
McCullough [2] (in particular. Lemma 12.1.2 of [2] is a version of Lemma 1.4 of [7] 
that applies to nonorientable 3 -manifolds). Corollary 0.3 fails in the nonorientable 
case, however. Not only can a nonorientable manifold with torus boundary admit Dehn 
twists about Mobius bands, but an annulus can meet the torus boundary in such a way 
that a Dehn twist about the annulus will be isotopic on the boundary torus to an even 
power of a simple Dehn twist about one of its boundary circles. 

Algebraic & Geometric Topology 6 (2006) 



Homeomorphisms which are Dehn twists on the boundary 1333 

Some of the work presented here is applied in the article Knot adjacency, genus and 
essential tori by EKalfagianni and X-SLin [5]. We are grateful to the authors of that 
paper for originally bringing the possibility of results like Theorem 1 and Theorem 2 to 
our attention. 

The research in this article was supported in part by NSF grant DMS-0102463. 



1 Proof of Theorem 1 

Recall that Dehn twists in 3-manifolds can be defined as follows. Consider first a 
properly imbedded and 2 -sided disk or annulus /^ in a 3 -manifold M. Imbed the 
product F X [0, 1] in M so that (F x [0, 1]) D dM = dF x [0, 1] and F x {0} = F. 
Let rg rotate F through an angle 6 (that is, if F is a disk, rotate about the origin, and 
if it is an annulus S^ x [0, 1], rotate in the 5 ^-factor). Fixing some integer n, define 
t: M ^ Mhy t(x) = x for x ^ F x [0, 1] and t{z, s) = (r2m«(z), s) if (z, s) £ F x [0, 1] . 
The restriction of t to dM is a Dehn twist about each circle of dF. Dehn twists are 
defined similarly when F is a 2-sphere or a two-sided projective plane, Mobius band, 
torus, or Klein bottle (for the case of tori, there are infinitely many nonisotopic choices 
of an 5^ -factor to define r^). Since a properly imbedded closed surface in M is disjoint 
from the boundary, a Dehn twist about a closed surface is the identity on dM. 

The proof of Theorem 1 will use the following result on Dehn twists about annuli in 
orientable 3-manifolds. 

Lemma 1.1 Let A\ and A2 be properly imbedded annuli in an orientable 3 -manifold 
M, with common boundary consisting of the loops C and C" . Let N' and N" be 
disjoint closed regular neighborhoods in dM of C' and C" respectively, and let ti be 
Dehn twists about the A,- whose restrictions to dM are supported on N' U A'" . If the 
restrictions of f 1 and t2 to N' are isotopic relative to dN' , then their restrictions to 
N" are isotopic relative to dN" . Consequently, if A is a properly imbedded annulus 
whose boundary circles are isotopic in dM (in particular, if they are contained in a torus 
boundary component of M), then any Dehn twist about A is isotopic to the identity 
on dM. 

Proof The result is clear if the A, have orientations so that their induced orientations 
on C' U C" are equal, since then the imbeddings of 5' x / x [0, 1] into M used to define 
the Dehn twists can be chosen to agree on S^ x dl x [0, 1] . So we assume that the 
oriented boundary of Ai is C' U C" and the oriented boundary of A2 is C' U (— C")- 

Algebraic & Geometric Topology 6 (2006) 



1334 Darryl McCullough 

By assumption, fi and t2 restrict to the same Dehn twist near C'. Their effects near C" 
differ in that after cutting along C" , the twisting of C" occurs in opposite directions, 
but since M is orientable, they also differ in that this twisting is extended to collar 
neighborhoods on opposite sides of C" (that is, the imbeddings of S^ x dl x [0, 1] used 
to define the Dehn twists fall on the same side of C' but on opposite sides of C"). Each 
of these differences changes a Dehn twist about C" to its inverse, so their combined 
effect is to give isotopic Dehn twists near C" . 

The last remark of the lemma follows by taking Ai = A and Ai to be an annulus with 
dA2 = dAi, with Ai parallel into dM. All Dehn twists about A2 are isotopic to the 
identity on dM, so the same is true for all Dehn twists about Ai . n 

We will also need a fact about homeomorphisms of reducible 3 -manifolds, even in 
many of the cases when M itself is irreducible. 

Lemma 1.2 Let W = P#Q be a connected sum of compact orientable 3 -manifolds, 
with P irreducible. Let S be the sum 2 -sphere. Suppose that dP is nonempty and that 
g: W —^ W is a homeomorphism which preserves a component of dP . Then there is a 
homeomorphism j : W ^ W , which is the identity on dW, such thatjg{S) = S. 

Proof Let Pi# ■ ■ ■ #Pr#R\# • • • #Rs be a prime factorization of W , where each P, is 
irreducible and each Rj is 5^ x 5' . Let S be the result of removing from a 3-sphere 
the interiors of r + 2^ disjoint 3-balls B\, . . . , Br, D\, E\, D2, . . . , E^. For 1 < / < r, 
let P'j be the result of removing the interior of a small open 3 -ball B'j from P,, and 
regard W as obtained from E and the union of the P'^ by identifying each dBi with dB'i 
and each dDj with dEj . 

In [7] and in Section 12.1 of [2], certain slide homeomorphisms of W are constructed. 
These can be informally described as cutting W apart along a dBi or dDj , filling in one 
of the removed 3-balls to obtain a manifold Y, performing an isotopy that slides that 
ball around a loop in the interior of Y, removing the 3 -ball and gluing back together to 
obtain a homeomorphism of the original W. Slide homeomorphisms are assumed to be 
the identity on dW (this is ensured by requiring that the isotopy that slides the 3 -ball 
around the loop in Y be the identity on dY at all times). Lemma 1.4 of [7], essentially 
due to M Scharlemann, says that if J is a collection of disjoint imbedded 2-spheres 
in the interior of W, then there is a composition 7 of slide homeomorphisms such that 
J{T) C S. 

Since P is irreducible, we may choose notation so that P = Pi and S = dB\ . Applying 
Lemma 1 .4 of [7] with T = g{S) , we obtain j so that jgiS) C S . In particular, there is a 
component ZofW —jg{S) whose closure contains P[ . Since g is assumed to preserve 
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a component of dPi , the closure of Z must be jgiP\). Since Pi is irreducible, jg{S) 
must be isotopic to 5 in VK, so changing j by isotopy we obtain jg{S) = S. n 

We can now prove Theorem 1. Let Nj be disjoint closed regular neighborhoods of the 
Cj in dM, and let F be the closure of dM — UjNj. By hypothesis, we may assume that 
h is the identity on F. Let M' be another copy of M, and identify F with its copy F' to 
form a manifold W with boundary a union of tori, each containing one C, . Denote by 
Tj the one containing Cj . Let g: W —^ W he h on M and the identity map on M' , so 
that on each Tj , g restricts to a nontrivial Dehn twist about Cj . 

Fix any C, , and for notational convenience call it Ci . Put Wi = W if W is irreducible. 
Otherwise, write W as Wi#W2 where W[ is irreducible and Ti C dWi , and let S be the 
sum sphere. By Lemma 1.2, there is a homeomorphism j of W that is the identity on 
dW, such that 7^(5) = S. Split W along S, fill in one of the resulting 2-sphere boundary 
components to obtain Wi and extend jg to that ball. This produces a homeomorphism 
gi of Wi that restricts on each boundary torus of Wi to a nontrivial Dehn twist about 
one of the Cj . 

Assume first that Wi has compressible boundary. Since Wi is irreducible, it is a solid 
torus with boundary T[ . The only nontrivial Dehn twists on Ti that extend to Wi are 
Dehn twists about a meridian circle, showing that Ci bounds a disk in W[ , and hence a 
disk E in W. Since Ci does not meet F, we may assume that E meets F transversely in 
a collection of disjoint circles. The intersection X' of E with M' has a mirror image X 
in M. Change E by replacing X' with X, producing a singular disk in M with boundary 
Ci . By the Loop Theorem, Ci bounds an imbedded disk in M. 

We call the argument in the previous paragraph that started with E in W and obtained a 
singular version of £ in M, having the same boundary as the original E, a swapping 
argument (since we are swapping pieces of the surface on one side of F for pieces on 
the other side). 

Suppose now that Wi has incompressible boundary. Let Vi be Johannson's characteristic 
submanifold of Wi ([4], also see Chapter 2 of [2] for an exposition of Johannson's 
theory). Since dWi consists of tori, Vi admits a Seifert fibering and contains all of 
dWi (in Johannson's definition, a component of Vi can be just a collar neighborhood 
of a torus boundary component). Each Cj in Wi is noncontractible in Tj, and Tj 
is incompressible in Wi , so Cj is noncontractible in Wi . This implies that Cj is 
noncontractible in W, hence also in M. 

It suffices to prove that Ci and some other C, cobound an imbedded annulus A in Wi 
and hence in W. For then, a swapping argument produces a singular annulus in M 
cobounded by Ci and C, . Since C[ and C, are noncontractible, a direct application of 
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the Generalized Loop Theorem [11] (see [3, p. 55]) produces an imbedded annulus in 
M cobounded by C\ and C, . 

By Corollary 27.6 of [4], the mapping class group of W\ contains a subgroup of finite 
index generated by Dehn twists about essential annuli and tori. So by raising gi to 
a power, we may assume that it is a composition of such Dehn twists. The Dehn 
twists about tori do not affect dWi , so we may discard them to assume that gi is a 
composition ?i • • • f„, , where each 4 is a Dehn twist about an essential annulus A^ ■ By 
Corollary 10.10 of [4], each A^ is isotopic into Vi . By Proposition 5.6 of [4], we may 
further change each A^ by isotopy to be either horizontal or vertical with respect to the 
Seifert fibering of Vi . 

Suppose first that some A^ is horizontal. Then Vi is either S^ x 5' x / or the twisted 
/-bundle over the Klein bottle (a horizontal annulus projects by an orbifold covering 
map to the base orbifold, and the orbifold Euler characteristic shows that the base 
orbifold is either an annulus, a Mobius band, or the disk with two order-2 cone points, 
the latter two possibilities yielding the two Seifert fiberings of the twisted /-bundle over 
the Klein bottle). In the latter case, dVi = Ti, so Wi = V\ and therefore dW[ = T\ . 
By Lemma 1.1, each tk is isotopic to the identity on T\ , hence so is gi , a contradiction. 
So Vi=SWs^ y. I. 

Since A^ is horizontal, it must meet both components of dV\ , and we have V\ = W\ 
and dW\ = Ji U J,- for some i. Let Aq = Ci x / c 5' x 5' x /. For an appropriate 
Dehn twist t about Aq, t^^g is isotopic to the identity on T\ . Using Lemma 3.5 of [12], 
f"'g is isotopic to a level-preserving homeomorphism of W\ , and hence to the identity. 
We conclude that g\ is isotopic to t, and consequently C\ and C, cobound an annulus 
in Wi. 

It remains to consider the case when all Aj. are vertical. In this case, each f^ restricts on 
dW\ to Dehn twists about loops isotopic to fibers, so each Cj in dWi is isotopic to a 
fiber of the Seifert fibering on V\ . 

Let V[ be the component of V\ that contains C\ . Suppose first that V[ n dW\ = T\ . 
Then each A^ that meets Ti has both boundary circles in T\ , so Lemma 1 . 1 implies that 
g\ is isotopic to the identity on Ji , a contradiction. So V[ contains another J,. Since 
Ci and Cj are isotopic to fibers, there is an annulus in V[ with boundary C\ U C;. 



2 Proof of Theorem 2 

Theorem 1 provides a properly imbedded surface S which is either an imbedded disk 
with boundary C„ or an incompressible annulus with boundary C„ and some other C; . 
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For some Dehn twist f„ about S, t„ and h are isotopic near C„. The composition ?,7'/z 
is isotopic on dM to a composition of Dehn twists about Ci , . . . , C„_ i (some of them 
possibly trivial). Induction on n produces a composition t as in the theorem, except for 
the assertion that the disks and annuli may be selected to be disjoint. 

Let D\, . . . , Dr and Ai , . . . , A^ be the disks and annuli needed for the Dehn twists in t. 
We first work on the annuli. 

We will say that a union A of disjoint incompressible imbedded annuli in M is sufficient 
for A 1 , . . . , Ajt if each boundary circle of ^ is isotopic in dM to a boundary circle of one 
of the Ai, and if for any composition of Dehn twists about the set (Uj^jD,) U (UJ^^jAy), 
there is a composition of Dehn twists about the union of Uj^jD, and the annuli of A 
which has the same effect, up to isotopy, on dM. In particular, A = A\ is sufficient 
for A\ alone. Inductively, suppose that A is sufficient for Ai, . . . , A^-i- By a 
routine surgery process, we may change A so that A^t and A intersect only in circles 
essential in both Ak and A. (First, make A transverse to Ak. An intersection circle 
which is contractible in A must also be contractible in Ayt , since both A and A^ are 
incompressible. If there is a contractible intersection circle, then there is a disk E in A^ 
with dE a component of Ayt n ,A and the interior of E disjoint from A. Replace the disk 
in A bounded by dE with E, and push off by isotopy to achieve a reduction of Ayt n^.) 

Now let Z be a closed regular neighborhood of A^ U ^. Since all intersection circles of 
Ak with A are essential in both intersecting annuli, each component of Z has a structure 
as an 5' -bundle in which the boundary circles of A and Ak are fibers. 

We will show that Z contains a collection sufficient for A^ U ,A and hence also for 
Ai, . . . , A^. We may assume that Z is connected. For notational simplicity, there is 
no harm in writing C\, . . . , Cm for the boundary circles of A and A^ , since they are 
isotopic in dM to some of the original C, . 

Fix a small annular neighborhood A^ of Ci in Zn dM. Using the 5^ -bundle structure 
of Z, we can choose a collection B2, . . . , B^ of disjoint annuli, with B, running from 
Ci to a loop in A'^ parallel to Ci . 

Consider one of the annuli A of A,t U ^, say with boundary circles isotopic to C, and Cj . 
If either / or 7 is 1 , say 7 = 1, then by Lemma 1.1, Dehn twists about A have the same 
effect on dM as Dehn twists about 6, . If neither is 1 , form an annulus B connecting C, 
to Cj by taking the union of Bj, Bj, and the annulus in A^ connecting Bj n N to Bj n N , 
then pushing off of A^ to obtain a properly imbedded annulus. Observe that any Dehn 
twist about B is isotopic on M to a composition of Dehn twists about B, and Bj . By 
Lemma 1 . 1 , there is a Dehn twist about B whose effect on dM is the same as the twist 
about A. This shows that the collection B2, ■ ■ ■ , B,„ is sufficient for Ai, . . . , A^^ and 
completes the induction. So there is a collection A sufficient for Ai, . . . , A^. 
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By further routine surgery, we may assume that each D, is disjoint from A. Then, 
surger D2 to make D2 disjoint from D\ , surger D3 to make it disjoint from Dy U D2, 
and so on, eventually achieving the desired collection of disjoint disks and annuli. 



3 Compression bodies 

Compression bodies were developed by FBonahon [1], in a study of cobordism of 
surface homeomorphisms. They were used in work on mapping class groups of 3- 
manifolds [9, 6, 8] and on deformations of hyperbolic structures on 3-manifolds [2]. 
The homeomorphisms of compression bodies were further investigated by Oertel [10], 
who develops an analogue for compression bodies of the Nielsen-Thurston theory of 
surface homeomorphisms. 

To fix notation and terminology, we recall that a compression body is a connected 
3 -manifold V constructed by starting with a compact surface G with no components 
that are 2-spheres, forming G x [0, 1], and then attaching 1 -handles to G x {1}. 
Compression bodies are irreducible. They can be handlebodies (when no component 
of G is closed) or product /-bundles (when there ai^e no 1 -handles). The exterior 
boundary of V is dV — (G x {0} U dG x [0, 1)). Note that if F is the exterior boundary 
of V , and A^ is a (small) regular neighborhood in V of the union of F with a collection 
of cocore 2-disks for the 1 -handles of V , then each component of V — A/^ is a product 
X X I, where X x {0} is a component of the frontier of A' and Z x { 1 } is a component 
of Gx {0}. 

The following result was proven in [10] for the case of V a handlebody. 

Corollary 3.1 Let V bea compact orientable compression body, and leth: V ^ V be 
a homeomorphism which is Dehn twists on the boundary about the collection Ci , . . . , 
C„ of simple closed curves in dV . Then h is isotopic to a composition of Dehn twists 
about a collection of disjoint disks and incompressible annuli in V, each of whose 
boundary circles is isotopic in dV to one of the Ci . 

To prove Corollary 3.1, we note first that by Theorem 2, there is a composition t of 
Dehn twists about a collection of disjoint disks and incompressible annuli in V, such 
that t and h are isotopic on dV. Changing h by isotopy, we may assume that t^^h is 
the identity on dV. Corollary 3.1 is then immediate from the following lemma. 

Lemma 3.2 Let V bea compression body with exterior boundary F , and let g: V ^ V 
be a homeomorphism which is the identity on F. Then g is isotopic relative to F to the 
identity. 
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Proof We have noted that there is a collection of disjoint properly imbedded disks 
E\, . . . ,E„, with boundaries in F, such that if A^ is a regular neighborhood of F U (UiEi) , 
then each component of V — A^ is a product X x I, where X x {0} is a component of 
the frontier of A'^. Now dEi is fixed by g, so we may assume that ^(£1) n £1 consists 
of dEi and a collection of transverse intersection circles. Since V is irreducible, we 
may change g by isotopy relative to F to eliminate these other intersection circles, and 
finally to make g fix E[ as well as F. Inductively, we may assume that g is the identity 
on F U (UiEj) and then on A'^. Finally, for each component Xxl of V — N,g is the 
identity on X x {0}. Using Lemma 3.5 of [12], g may be assumed to preserve the 
levels X X {s} oi X x I, and then there is an obvious isotopy from g to the identity on 
Xxl, relative to Z x {0} . Applying these isotopies on the complementary components 
of A'^, we make g the identity on V. D 
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